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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )E3 Responsive to communication(s) filed on 21 October 2003 . 
2a)Q This action is FINAL. 2b)G3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) 1 and 10-18 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [3 Claim(s) 2-9, 19 and 20 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)dAII b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1.78. 
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DETAILED ACTION 
Response to Amendment 

1. Applicant's arguments with respect to claims 2, 5, 9, 19, 20 have been considered but are 
moot in view of the new ground(s) of rejection. Claims 1, 10-18 are cancelled. New claim 
number 20 is received. Claims 2-9, 19, 20 are pending. 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless -(a) the invention was known or used by others in this country, 
or patented or described in a printed publication in this or a foreign country, before the invention thereof by 
the applicant for a patent. 

3. Claims 2-4, 6-8, 19, 20 are rejected under 35 U.S.C. 102(a) as being anticipated by 
Maclnnis, et al, U.S. Patent 6,501,480. 

4. Regarding claim 2, representative of claim 20, Maclnnis discloses a device for producing 
a composite (Column 3, line 51) graphical [digital video, Column 1, lines 49-60] data stream 
(Figure 1, Column 1, lines 50-53) containing pixel data (Column 5, lines 2, 5, 31) corresponding 
to an image to be rendered (Figures 1-5, Column 1 1, lines 16-20), the composite graphical 
[digital video] data stream being formed from multiple graphical [digital video] data streams 
(Column 3, lines 39-41, Column 4, lines 1-16; Column 6, lines 30-45), each of the multiple 
graphical [digital video] data streams being provided by a graphics pipeline (Figure 4, Column 6, 
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lines 19-21), each graphics pipeline being configured to process pixel data corresponding to at 
least a portion of the image to be rendered, said device comprising: an input mechanism 
configured to receive the multiple graphical [digital video] data streams from the graphics 
pipelines, provide a frame of data corresponding to the image to be rendered, and insert pixel 
data from the multiple graphical data [digital video] streams into said frame of data such that, in 
response to receiving a first of the multiple graphical [digital video] data streams, said input 
mechanism provides said frame of data and inserts the pixel data from the first of the multiple 
graphical [digital video] data streams into said frame of data to form at least a portion of the 
composite graphical [digital video] data stream (Figure 5, Column 9, line 5 through Column 11, 
line 7; Column 43, lines 62 through Column 47, line 45; Column 8, line 61 through Column 9, 
line 4; Column 43, line 62 through Column 44, line 33, ... blending of different layers of graphics 
and/or video); wherein said input mechanism has a first compositing element and a second 
compositing element, said first compositing element being configured to provide said frame of 
data in response to receiving pixel data corresponding to the first of the multiple graphical 
[digital video] data streams {MPEG or other digital video signals input to the system, analog 
video signals, digital video signals, Column 3, lines 39-55; With both analog and digital video 
input, either one may be scaled while the other is displayed full size at the time as passthrough 
video. Column 4, lines 34-36; various numbers of signals may be composited, including, for 
example two or more video windows, Column 10, lines 65-66; Column 9, lines 60-62), said first 
compositing element being further configured to insert the pixel data corresponding to the first of 
the multiple graphical [digital video] data streams into said frame of data to form a first 
compositing graphical [digital video] data stream, said pixel data corresponding to the first of the 
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multiple graphics [digital video] data streams and a first displayed portion of said image, said 
second compositing element being configured to receive pixel data corresponding to the second 
of the multiple graphical [digital video] data streams and said first compositing graphical [digital 
video] data stream, said pixel data corresponding to the second of the multiple graphics [digital 
video] data streams and a second displayed portion of said image, said second compositing 
element (graphics compositor engine, Column 47, line 25; Figure 6, element 58, display engine, 
Column 5, lines 24-50) being further configured to combine the pixel data corresponding to the 
second of the multiple graphical [digital video] data streams and said first compositing graphical 
[digital video] data stream to form a second compositing digital video data stream (Figure 4, 
element 108, video compositor block with three inputs: graphics display, video display and 
passthrough video, Column 8, lines 61-63; Column 1, lines 49-60). 

5. Regarding claim 3, Maclnnis discloses the device of claim 2, wherein the multiple 
graphical data streams simultaneously (Column 4, lines 34-36) provide pixel data to said input 
mechanism, the first of the multiple graphical data streams containing three-dimensional 
(Column 5, line 53) pixel data corresponding to the image to be rendered, the second of the 
multiple graphical data streams containing two-dimensional (Column 11, lines 60 through 
Column 12, line 14) pixel data corresponding to the image to be rendered, and wherein said input 
mechanism is configured to combine said two-dimensional pixel data and said three-dimensional 
pixel data by replacing at least a portion of the pixel data provided by the second of the multiple 
graphical data streams with at least a portion of the pixel data provided by the first of the 
multiple graphical data streams (Column 57, lines 3, 6, 15, 17). 
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6. Regarding claim 4, Maclnnis discloses the device of claim 2, further comprising: a 
controller electrically communicating with said input mechanism (Figures 2-4, Column 4, line 1 
through Column 6, line 20) said controller being configured to provide a first control signal to 
said input mechanism, said first control signal containing information regarding which portion of 
said frame of data corresponds to the pixel data provided from the first of the multiple graphical 
data streams such that, in response to receiving said first control signal and the pixel data from 
the first of the multiple graphical data streams, said input mechanism inserts the pixel data from 
the first of the multiple graphical data streams into said corresponding portion of said frame of 
data to form at least a portion of the composite graphical data stream (Column 4, lines 51-67). 

7. Regarding claim 6, representative of claim 8, Maclnnis discloses the device of claim 2, 
wherein said controller is configured to provide a second control signal to said input mechanism, 
said second control signal corresponding to one of multiple compositing modes (Column 4, lines 
34-39), a first of said compositing modes corresponding to each of the graphics pipelines 
providing pixel data associated with an entire frame of the image to be rendered, the pixel data of 
each of the graphics pipelines including a coordinate value offset (Column 111, lines 9-20; 
Column 12, line 11; Column 13, lines 65; Column 14, line 51) with respect to pixel data of 
others of the graphics pipelines, said input mechanism being configured to combine the pixel 
data from the multiple graphical data streams so as to blend color values associated with 
corresponding coordinate values (Column 30, line 54-55). 



Page 5 
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8. Regarding claim 7, Maclnnis discloses the device of claim 2, wherein said controller is 
configured to provide a second control signal to said input mechanism, said second control signal 
corresponding to one of multiple compositing modes, a first of said compositing modes 
corresponding to each of the graphics pipelines providing pixel data associated with a portion of 
the image to be rendered, the pixel data of each of the graphics pipelines being super sampled, 
said input mechanism being configured to average, with a selected weighting, the super-sampled 
pixel data (see above; Column 30, lines 32 through Column 31, line 36; through line 43, 
antialiased text and graphics). 

9. Regarding claim 19, Maclnnis discloses the device of claim 2, wherein said controller is 
configured to provide a second control signal, a third control signal and a fourth control signal 
selectively to said input mechanism; said second control signal corresponding to a second of 
multiple compositing modes, the second of said compositing modes corresponding to each of the 
graphics pipelines providing pixel data associated with an entire frame of the image to be 
rendered, the pixel data of each of the graphics pipelines including a coordinate value offset 
(changing the start address... shifts of any number of pixels, Column 29, lines 50-56) with 
respect to pixel data of others of the graphics pipelines, said input mechanism being configured 
to combine the pixel data from the multiple graphical data streams so as to blend color values 
associated with corresponding coordinate values (pixel color type, alpha blend factor, location 
on the screen, Column 4, lines 65-67); said third control signal corresponding to a third of 
multiple compositing modes, a first of said compositing modes corresponding to each of the 
graphics pipelines providing pixel data associated with a portion of the image to be rendered, the 



Application/Control Number: 09/7 1 5,892 Page 7 

Art Unit: 2676 

pixel data of each of the graphics pipelines being super sampled (Column 31, lines 19-43), said 
input mechanism being configured to average, with a selected weighting, the super sampled pixel 
data; said fourth control signal corresponding to a fourth of multiple compositing modes, the 
fourth of said compositing modes corresponding to each of the graphics pipelines providing pixel 
data associated with a portion of the image to be rendered, said input mechanism being 
configured to combine the pixel data from the multiple graphical data streams to form the 
composite graphical data stream, {transparent black, video passthrough, digital video, analog 
video, upscale, downscale, graphics data simultaneously, Column 43, lines 63-67; Column 42, 
lines 31-39; Column 5, lines 24-50; Figure 5, Column 9, line 5 through Column 11, line 7; 
Column 43, lines 62 through Column 47, line 45; Column 8, line 61 through Column 9, line 4; 
Column 43, line 62 through Column 44, line 33, ... blending of different layers of graphics and/or 
video', MPEG or other digital video signals input to the system, analog video signals, digital 
video signals, Column 3, lines 39-55; With both analog and digital video input, either one may 
be scaled while the other is displayed full size at the time as passthrough video. Column 4, lines 
34-36; blends signals from four different sources, ...various numbers of signals may be 
composited, including, for example two or more video windows, Column 10, lines 60-66; 
Column 9, lines 60-62; graphics compositor engine, Column 47, line 25; Figure 6, element 58, 
display engine, Column 5, lines 24-50; Column 42, lines 31-39; video signals are processed in 
the video display pipeline while the graphics data is processed in the graphics display pipeline 
[each}, respectively, and outputs to...], Column 43, line 63 through Column 44, lines 7; two or 
more graphics windows may be processed in parallel, Column 6, lines 37-40; logical windows, 
Column 4, lines 51-61). 
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Claim Rejections - 35 USC § 103 



10. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 



11. Claims 5, 9 are rejected under 35 U.S.C. 103(a) as being unpatentable over Maclnnis, et 
al, U.S. Patent 6,501,480, in view of Jones et al, U.S. Patent 6,573,928. 



Application/Control Number: 09/7 1 5,892 Page 9 

Art Unit: 2676 

12. Regarding claim 5, Maclnnis discloses the device of claim 20, further comprising: an 
output mechanism electrically communicating with said input mechanism, said output 
mechanism being configured to receive the composite digital video data stream and provide an 
output composite video data stream, said output composite video data stream being selectively 
configurable as any one of an analog video data stream, an analog video data stream, a digital 
video data stream, and a digital video data stream (Figures 1-3, Column 1, lines 39-55; Column 
4, lines 1-29). 

Maclnnis does not disclose stereo. Jones teaches stereo (Column 9, lines 63-67). The 
motivation for combining composited graphics and video image data with stereo data is enable 
3D vision without adding extra memory (Column 6, lines 50-54). Jones is evidence that at the 
time of invention it would have been obvious to one skilled in designing 3D video graphics 
machines to combine the benefits of parallel graphics and video compositing, as Maclnnis 
discloses, with stereoscopic vision methods, as Jones teaches, to provide 3D vision without 
adding extra memory. 

13. Regarding claim 9, Maclnnis discloses the device of claim 5, wherein said output 
mechanism has a first left channel frame buffer, a second left channel frame buffer, a first right 
channel frame buffer, and a second right channel frame buffer, said output mechanism being 
selectively configured to provide said passive digital video data stream by receiving said 
composite digital video data stream, allocating pixel data from said composite digital video data 
stream to said first left channel frame buffer, said second left channel frame buffer, said first 
right channel frame buffer, and said second right channel frame buffer, and simultaneously 
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outputting pixel data from one of said left channel frame buffers and one of said right channel 
frame buffers (Figures 2-5, elements 52, 59, 60, 184, 186; Figure 14, Column 25, line 62 through 
Column 30, line 31). 

Maclnnis does not disclose stereo. Jones teaches stereo (Column 9, lines 63-67). The 
motivation for combining composited graphics and video image data with stereo data is enable 
3D vision without adding extra memory (Column 6, lines 50-54). Jones is evidence that at the 
time of invention it would have been obvious to one skilled in designing 3D video graphics 
machines to combine the benefits of parallel graphics and video compositing, as Maclnnis 
discloses, with stereoscopic vision methods, as Jones teaches, to provide 3D vision without 
adding extra memory. 

Response to Arguments 

14. Applicant's arguments with respect to claims 2, 5, 9, 19, 20 have been considered but are 
moot in view of the new ground(s) of rejection. 

15. Applicant asserts (Page 1 1, 3 rd Paragraph; Page 13, 3 rd Paragraph) that the 
compositing/blending in Maclnnis is not "from a frame of data generated by a compositing 
element that includes pixel data corresponding to separate displayed portions of an image". 

Examiner respectfully notes, however, that, in the claims, Maclnnis discloses frame- 
based compositing of pixels corresponding to separate displayed portions of an image (Column 
5, line 24 through Column 6, line 67, graphics blending, next window, two or more graphics 
windows; Column 11, lines 52-55; multiple compositors: a video compositor (Column 8, lines 
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61-67) , a graphics compositor engine ...composited in other line buffers. (Column 47, line 25- 
39), upper layers are composited in memory buffer storage buffers called line buffers. Each line 
buffer... is sized to contain pixels for one scan line (column 46, lines 43-45). 

16. Applicant asserts (Page 12, 3 rd Paragraph through Page 13, 1 st Paragraph) a 
distinguishment between video image data and graphics image data, and that Maclnnis is 
compositing a video signal with a graphics signal. 

The Examiner respectfully replies however, that, in the claims, Maclnnis clearly 
addresses the matter, thus: "The display engine is the part of the graphics display system that 
receives display pixel data from any combination of locally attached video and graphics input 
ports, processes the data in some way, and produces final display pixels as output. This 
application includes references to both graphics and video, which reflects in certain ways the 
structure of the hardware itself. This split does not, however, imply the existence of any 
fundamental difference between graphics and video, and in fact much of the functionality is 
common to both. Graphics is used herein may include graphics, text and video (Column 1 , lines 
45-60)." 

Furthermore, Maclnnis discloses compositing/blending graphics data streams (After the 
processing of the video signals and the graphics data have been completed... receives... data 
from... the graphics display pipeline, respectively, blending a plurality of graphics images into a 
blended graphics image, blending of different layers of graphics and/or video, Columns 44, lines 
1-38; each pixel, reading pixels from memory, as in a conventional graphics display device, 
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Column 45, line 64 through 46, line 55; the graphics compositor engine, Column 47, lines 24-37; 
Figure 5, element 140, Figure 28, element 904). 

17. Applicant asserts (Page 14, 3 rd Paragraph) that Maclnnis does not disclose stereo video. 
The Examiner respectfully must agree in that explicitly the term "stereo" is not there, but the 
Examiner carefully notes that Maclnnis uses the terms scalability, layered depth, warping, wich 
are well-documented stereo vision/3D technical MPEG and HDTV compliance issues, which 
Maclnnis is addressing in his work. Nonetheless, the Examiner has found an additional 
reference, Jones, that teaches stereo. And, the Examiner has provided other references as prior 
art not used to clarify and document that these features disclosed by Maclnnis are in fact 
stereo/3D TV issues involving MPEG and HDTV compliance. 



Maclnnis et al, U.S. Patent 6,573,905, multiple graphics layers for HDTV and MPEG 

Szeliski, U.S. Patent 6,487,304, Multiview Approach to Motion and Stereo, layered 
approach to stereo applications in a networked environment. 

Buckelew et al, U.S. Patent 6,278,645, Frame-sequential stereo using pixel planes, and 
using regions of the display. 

Popescu, Voicu, et al, Efficient Warping for Architectural Walkthroughs Using Layered 
Depth Images, IEEE, @1998. 



Prior Art Not Used 
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Ho-Chao Huang and Yi-Ping Hung, "Panoramic Stereo Imaging System with Automatic 
Disparity Warping and Seaming", Graphics Models and Image Processing, Vol. 60 No. 3 May, 
pp. 196-208, 1998. 

J. Shade, S. Gortler, L.-W. He, and R. Szeliski, Layered Depth Images , Computer 
Graphics (SIGGRAPH *98), Proceedings, pp. 231-242, Orlando, FL, July, 1998. 



18. Applicant and the assignee of this application are required under 37 CFR 1 . 1 05 to 
provide the following information that the examiner has determined is reasonably necessary to 
the examination of this application. Regarding the other basis of rejections, Examiner 
respectfully notes that the copending applications, 09/715,253, '232 (now US Patent 6,680,739 
to be effective January 20, 2004), '746, '882 (now US Patent 6,621,500, issued September 16, 
2003), '600, '335, as revealed by the Applicant in the IDS, still stand under the double patenting 
statues, as noted in the Office Action dated February 12, 2003. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Allen E. Quillen whose telephone number is (703) 605-4584. 
The examiner can normally be reached on Tuesday - Friday, 8:30am - noon and 1 :00 - 4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew C. Bella, can be reached on (703) 308-6829. 

Any response to this action should be mailed to: 



Requirements for Information 



Conclusion 



* 
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Commissioner of Patents and Trademarks 



Washington, D C. 20231 



Or FAX'd to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, Sixth 
Floor (Receptionist), Arlington, Virginia 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number (703) 305-9600 or (703) 305- 
3800. 



Allen E. Quillen 
Patent Examiner 
Art Unit 2676 



January 2, 2004 




